Skip to content

[Core][SM70] Promote DFlash2 NVFP4 prefill route - #417

Merged
yangzhuxinyzx merged 2 commits into
mainfrom
agent/v100-dflash2-nvfp4-prefill-promotion-20260829-042116
Aug 29, 2026
Merged

yangzhuxinyzx merged 2 commits into
mainfrom
agent/v100-dflash2-nvfp4-prefill-promotion-20260829-042116

Conversation

@yangzhuxinyzx

@yangzhuxinyzx yangzhuxinyzx commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Purpose

Automatically select the existing lossless QPN2-packed bounded prefill path for the exact Qwen3.8-27B NVFP4 DFlash2 TP4 single-request contract. The default gate requires method=dflash, q7, checkpoint selector top-K 16, PP1/TP4, B1, no DBO/microbatching, and the exact dense layer shapes. Explicit environment overrides remain hard rollbacks; DFlash1, target-only, other speculative methods, TP sizes, and concurrent service retain existing routing.

This PR changes dispatch only. It does not change kernel arithmetic, DFlash2 proposal/verification, sampling, attention, prefix-cache behavior, or the merged tool/structured-output fixes.

BASE_SHA: 62ad1e0

Test Plan

  • Focused resolver, DFlash1/DFlash2 selector isolation, exact-shape, zero-copy-layout, and dispatch tests.
  • Latest-main TP4 V100 route-hit and cold 32K/64K prefill measurements under the production NVFP4 DFlash2 contract.
  • Preserve the merged QPN2 decode/quality contract and verify long-prefill output integrity against retained artifacts.

Test Result

  • Focused CPU suite: 5 passed.
  • Ruff check/format and git diff --check: passed.
  • Four-V100 latest-main route hit: automatic QPN2 M<=8, automatic QPN2-packed prefill M>=1024, fixed-SHA D256 sidecar, direct paged chunked prefill, FP8 E5M2 bridge, and DFlash2 context-only chunk skip all active.
  • 32K: 8.0618 s request wall, 4064.62 request-wall tok/s, 4069.25 pure-prefill tok/s.
  • 64K: 18.3891 s request wall, 3563.85 request-wall tok/s, 3566.94 pure-prefill tok/s.
  • Versus retained pre-closure DFlash2: +30.1% at 32K and +37.7% at 64K. No target-only rerun was performed.
  • All six measured requests were uncorrupted and produced token ID 248046 with retained output SHA256 54363ddee68f4a5db81c9d37e5fb738d28f5b67dc7f725ad7333172b1ea157da.
  • The unchanged QPN2 decode path was already covered by merged PR [Bugfix][SM70] Audit DFlash2 output quality #346: structured API 24/24, long alternating-prefix state 5/5, three-seed MBPP/HumanEval/LiveCodeBench gates, and target-only/DFlash2 PPL 5.4993116/5.4993622.

Raw artifacts and exact hashes are recorded in docs/design/sm70_dflash2_nvfp4_prefill_promotion.md.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

Four-V100 production-service gate passed on head a09e332: health/models HTTP 200; automatic QPN2 M<=8 and QPN2-packed M>=1024 routes hit with both env gates unset; D256/direct-paged/FP8-bridge/context-only prefill routes hit; qwen3_coder tool call returned structured finish_reason=tool_calls with valid get_weather({"city":"广州"}); repeated ~8K prefix reduced TTFB 2.789s -> 0.591s with 9,888 local prefix-cache hit tokens. The public tunnel was also verified. Old service remains disabled as rollback.

@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 29, 2026 05:11
@yangzhuxinyzx
yangzhuxinyzx merged commit cda10f1 into main Aug 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant